The present invention is in the technical field of architectures for computational devices. More specifically, the present invention is in the field of pattern detection and feature learning.
This section is intended to provide a background or context to the invention disclosed below.
The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived, implemented or described. Therefore, unless otherwise explicitly indicated herein, what is described in this section is not prior art to the description in this application and is not admitted to be prior art by inclusion in this section.
Neuromorphic systems provide a biologically inspired method of computing as an alternative to the classical von Neumann approach. The classical von Neumann architecture is increasingly being challenged by the throughput limitations between the physically separated memory and the CPU. Other challenges arise from the exponential growth of large streams of unstructured data, as von Neumann machines are often inefficient in terms of power or area or are too slow for a large class of parallel workloads.
Neuromorphic computing explores biologically inspired, energy-, power density-, and area-efficient hardware architectures capable of learning and of carrying out event-based computations. In neuromorphic systems, computation is performed by a network of spiking neurons controlled by the values of their synaptic weights, which are obtained in the process of learning. In contrast to the classical von Neumann architecture, a neuromorphic system implementing a spiking neural network (SNN) is inherently parallel: neurons perform operations asynchronously, and synapses receive operands from a network of synaptic interconnections. Furthermore, owing to developments in the field of nanomaterials, further benefits in area efficiency and power dissipation can be gained by using memristive devices for the neurosynaptic realization. Specifically, neurons and synapses can be implemented using memristive circuit elements in which the device conductance represents the membrane potential and the synaptic weight. Synaptic learning occurs based on the currents flowing through the devices.
Providing efficient synaptic learning rules is a challenging task. From the implementation aspect, these rules should primarily use local information, as in spike-timing dependent plasticity (STDP), but simultaneously develop a knowledge representation that is useful in the global context. For example, in the case of multi-layer artificial neural networks, feature-based representations develop, in which neurons become sensitive to certain informative characteristics of the input, and enable increasingly abstract processing to occur in the consecutive layers.
The values of the synaptic weights represent the knowledge of a network, which determines its performance on a particular task. Numerous approaches exist for learning neural networks, but the lowest error rates are obtained in deep learning, where an artificial neural network (ANN) learns a hierarchy of features, e.g., certain useful characteristics of an input. Feature learning is an approach to learning neural networks, in which extracting informative properties of the input is preferred to memorizing all possible input patterns. These properties, called features, are then utilized by multi-layered architectures to improve the accuracies on difficult tasks. In contrast, SNNs are more suitable for efficient event-based hardware realizations. The knowledge from an ANN can be translated into an SNN by porting the weights. However, it would be beneficial to develop an architecture that learns certain features directly in a spiking network.
Neural networks are also important in achieving unsupervised and self-organizing sub-systems. Large scale lateral inhibition can give rise to competitive or “winner-take-all” behavior within an area defined by the range of the inhibition. When excitatory connections are also used, topographic or computational mapping can be established. This is because if a particular node responds to a given input, nearby nodes may also partly respond. There is extensive evidence from neuroscience as to the importance of topographic mappings and indeed it is regarded as being a building block in the infrastructure of information processing in the nervous system.